Minimum complexity density estimation

The minimum complexity or minimum description-length criterion developed by Kolmogorov, Rissanen, Wallace, So&in, and others leads to consistent probability density estimators. These density estimators are defined to achieve the best compromise between likelihood and simplicity. A related issue is the compromise between accuracy of approximations and complexity relative to the sample size. An index of resolvability is studied which is shown to bound the statistical accuracy of the density estimators, as well as the informationtheoretic redundancy. Index Terms -Kolmogorov complexity, minimum description-length criterion, universal data compression, bounds on redundancy, resolvability of functions, model selection, density estimation, discovery of probability laws, consistency, statistical convergence rates.